Effective and feasible interventions to improve structured EHR data registration and exchange: A concept mapping approach and exploration of practical examples in the Netherlands.

BACKGROUND: Data in Electronic Health Records (EHRs) is often poorly structured and standardized, which hampers data reuse. Research described some examples of interventions to increase and improve structured and standardized data, such as guidelines and policies, training and user friendly EHR interfaces. However, little is known about the translation of this knowledge into practical solutions. Our study aimed to specify the most effective and feasible interventions that enable better structured and standardized EHR data registration and described practical examples of successfully implemented interventions. METHODS: A concept mapping approach was used to determine feasible interventions that were considered to be effective or have been successfully implemented in Dutch hospitals. A focus group was held with Chief Medical Information Officers and Chief Nursing Information Officers. After interventions were determined, multidimensional scaling and cluster analysis were performed to categorize sorted interventions using Groupwisdom™, an online tool for concept mapping. Results are presented as Go-Zone plots and cluster maps. Following, semi-structured interviews were conducted to describe practical examples of successful interventions. RESULTS: Interventions were classified into seven clusters ranked from highest to lowest perceived effectiveness: (1) education on usefulness and need; (2) strategic and (3) tactical organizational policies; (4) national policy; (5) monitoring and adjusting data (6) structure of and support from the EHR and (7) support in the registration process (EHR independent). Interviewees emphasized the following interventions proven successful in their practice: an enthusiastic ambassador per specialty who is responsible for educating peers by increasing awareness of the direct benefit of structured and standardized data registration; dashboards for continuous feedback on data quality; and EHR functionalities that support (automating) the registration process. CONCLUSIONS: Our study provided a list of effective and feasible interventions including practical examples of interventions that have been successful. Organizations should continue to share their best practices to learn from and attempted interventions to prevent implementation of ineffective interventions.

A review of AI and Data Science support for cancer management.

INTRODUCTION: Thanks to improvement of care, cancer has become a chronic condition. But due to the toxicity of treatment, the importance of supporting the quality of life (QoL) of cancer patients increases. Monitoring and managing QoL relies on data collected by the patient in his/her home environment, its integration, and its analysis, which supports personalization of cancer management recommendations. We review the state-of-the-art of computerized systems that employ AI and Data Science methods to monitor the health status and provide support to cancer patients managed at home. OBJECTIVE: Our main objective is to analyze the literature to identify open research challenges that a novel decision support system for cancer patients and clinicians will need to address, point to potential solutions, and provide a list of established best-practices to adopt. METHODS: We designed a review study, in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, analyzing studies retrieved from PubMed related to monitoring cancer patients in their home environments via sensors and self-reporting: what data is collected, what are the techniques used to collect data, semantically integrate it, infer the patient's state from it and deliver coaching/behavior change interventions. RESULTS: Starting from an initial corpus of 819 unique articles, a total of 180 papers were considered in the full-text analysis and 109 were finally included in the review. Our findings are organized and presented in four main sub-topics consisting of data collection, data integration, predictive modeling and patient coaching. CONCLUSION: Development of modern decision support systems for cancer needs to utilize best practices like the use of validated electronic questionnaires for quality-of-life assessment, adoption of appropriate information modeling standards supplemented by terminologies/ontologies, adherence to FAIR data principles, external validation, stratification of patients in subgroups for better predictive modeling, and adoption of formal behavior change theories. Open research challenges include supporting emotional and social dimensions of well-being, including PROs in predictive modeling, and providing better customization of behavioral interventions for the specific population of cancer patients.

Standardization of immunotherapy adverse events in patient information leaflets and development of an interface terminology for outpatients' monitoring.

INTRODUCTION: Immunotherapy is effective for treating cancer, but it is also associated with a wide spectrum of adverse events. In order to detect them early, the patients need to be monitored at home, between the therapy administrations, e.g., by asking them to report outcomes, usually including symptoms and quality of life measures. For the collected data to be reusable, the symptoms need to be in a standardized form. The aim of this study is to explore the standardization of the information contained in the patient information leaflets (PILs) of immunotherapy drugs, by creating an interface terminology of immunotherapy-related adverse events, which should support a consistent collection of symptoms from the patients. METHODS: PILs contain a significant amount of information in free text, but they mix patient-reportable and clinically assessable events. We extracted a list of patient-reportable adverse events, mapped them to reference terminologies and compared the mapping results to choose the best-performing reference terminology. RESULTS: The PILs standardization led to the extraction of 151 symptoms and 424 terms, including both preferred terms and synonyms in English and Italian. Among the reference terminologies we considered, SNOMED CT allowed us to map all concepts and became, hence, the main reference terminology for the resulting interface terminology. A preliminary validation on the PIL of a new immunotherapy drug showed that our interface terminology already contained all the mentioned symptoms. CONCLUSION: PILs provide a valuable source for determining adverse events. The resulting interface terminology includes Italian and English terms for patient-reportable adverse events for five immunotherapy drugs representative of their category. Further work will be undertaken to evaluate the usability of the interface terminology and the patients' experience and satisfaction with the proposed terms, made available for example through an app, as well as its effectiveness on data quality and quality of care.

Health Concept and Knowledge Management: Twenty-five Years of Evolution.

OBJECTIVES: The fields of health terminology, classification, ontology, and related information models have evolved dramatically over the past 25 years. Our objective was to review notable trends, described emerging or enabling technologies, and highlight major terminology systems during the interval. METHODS: We review the progression in health terminology systems informed by our own experiences as part of the community involved in this work, reinforced with literature review and citation. RESULTS: The transformation in size, scope, complexity, and adoption of health terminological systems and information models has been tremendous, on the scale of orders of magnitude. CONCLUSION: The present "big science" era of inference and discovery in biomedicine would not have been possible or scalable absent the growth and maturation of health terminology systems and information models over the past 25 years.

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care.

OBJECTIVE: To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System. DESIGN: Observational study with user-based usability evaluations before and 3 months after the system was implemented and routinely used. MEASUREMENTS: Usability was defined by five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. Qualitative (the Think-Aloud user testing method) and quantitative (the System Usability Scale questionnaire and Time-on-Task analyses) methods were used to examine these usability aspects. RESULTS: The results of the evaluation study revealed that the usability of the interface terminology fell short (SUS scores before and after implementation of 47.2 out of 100 and 37.5 respectively out of 100). The qualitative measurements revealed a high number (n=35) of distinct usability problems, leading to ineffective and inefficient registration of reasons for admission. The effectiveness and efficiency of the system did not change over time. About 14% (n=5) of the revealed usability problems were related to the terminology content based on SNOMED CT, while the remaining 86% (n=30) was related to the terminology application. The problems related to the terminology content were more severe than the problems related to the terminology application. CONCLUSIONS: This study provides a detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology are defining for its usability. Carefully crafted domain-specific subsets and a well-designed terminology application are needed to facilitate the use of a complex compositional interface terminology based on SNOMED CT.

Construction of an interface terminology on SNOMED CT. Generic approach and its application in intensive care.

OBJECTIVE: To provide a generic approach for developing a domain-specific interface terminology on SNOMED CT and to apply this approach to the domain of intensive care. METHODS: The process of developing an interface terminology on SNOMED CT can be regarded as six sequential phases: domain analysis, mapping from the domain concepts to SNOMED CT concepts, creating the SNOMED CT subset guided by the mapping, extending the subset with non-covered concepts, constraining the subset by removing irrelevant content, and deploying the subset in a terminology server. RESULTS: The APACHE IV classification, a standard in the intensive care with 445 diagnostic categories, served as the starting point for designing the interface terminology. The majority (89.2%) of the diagnostic categories from APACHE IV could be mapped to SNOMED CT concepts and for the remaining concepts a partial match was identified. The resulting initial set of mapped concepts consisted of 404 SNOMED CT concepts. This set could be extended to 83,125 concepts if all taxonomic children of these concepts were included. Also including all concepts that are referred to in the definition of other concepts lead to a subset of 233,782 concepts. An evaluation of the interface terminology should reveal what level of detail in the subset is suitable for the intensive care domain and whether parts need further constraining. In the final phase, the interface terminology is implemented in the intensive care in a locally developed terminology server to collect the reasons for intensive care admission. CONCLUSIONS: We provide a structure for the process of identifying a domain-specific interface terminology on SNOMED CT. We use this approach to design an interface terminology on SNOMED CT for the intensive care domain. This work is of value for other researchers who intend to build a domain-specific interface terminology on SNOMED CT.

Design of polypeptide-functionalized polystyrene microspheres.

In this contribution, the principle of spontaneous surface segregation has been applied for the preparation of polypeptide-functionalized polystyrene microspheres. For that purpose, an amphiphilic diblock copolymer was introduced in the mixture styrene/divinylbenzene and polymerized using AIBN as initiator. During the polymerization, cross-linked particles were obtained in which the diblock copolymer was encapsulated. The amphiphilic diblock copolymers used throughout this study contain a hydrophilic polypeptide segment, either poly(L-lysine) or poly(L-glutamic acid) and a hydrophobic polystyrene block. After 4 h of polymerization, rather monodisperse particles with sizes of approximately 3-4 microm were obtained. Upon annealing in hot water, the hydrophilic polypeptides migrate to the interface, hence, either positively charged or neutral particles were obtained when poly(L-lysine) is revealed at the surface and exposed to acidic or basic pH, respectively. On the opposite, negatively charged particles were achieved in basic pH water by using poly(L-glutamic acid) as additive. The surface chemical composition was modified by changing the environment of the particles. Thus, exposure in toluene provoked a surface rearrangement, and due to its affinity, the polystyrene block reorients toward the interface.

Cross-mapping APACHE IV "reasons for intensive care admission" classification to SNOMED CT.

The APACHE IV classification is used to capture diagnostic information for calculation of mortality risks in intensive care (IC). The lack of structured and formal definitions for concepts in APACHE IV classification as in any classification results in shortcomings when scaling up for re-use. The use of SNOMED CT as a reference terminology can address these shortcomings. However, all of SNOMED CT contains large amounts of information that is irrelevant for IC. By building an interface terminology (IfT) based on SNOMED CT and APACHE IV, it is possible to isolate the IC users from the complexity of SNOMED CT while enabling standardized data registration. Within this study, a mapping is realized from the APACHE IV classification to SNOMED CT. The results of the mapping will be used to identify a relevant SNOMED CT subset for the development of an IC-specific IfT. The vast majority of the diagnostic categories in APACHE IV could be mapped to one or more SNOMED CT concepts (83.8%) and for the remaining concept a partial match was identified (16%). The good mapping results will provide a SNOMED CT subset sufficient for developing an IC-specific IfT. Finally, lessons learned in this study are valuable for other researchers who intend to realize a mapping from a classification to SNOMED CT.

Functional crosslinked polymer particles synthesized by precipitation polymerization for liquid chromatography.

Highly crosslinked functional polymer particles with narrow size distribution have been produced by precipitation copolymerization of divinylbenzene, ethylene glycol dimethacrylate and vinylbenzyl chloride using a simple reflux protocol. After establishing the satisfactory synthesis conditions, we produced uniform chlorobenzyl particles with different size depending on the polymerization times. The porosity of those particles was modulated from microporous to mesoporous structure by using various porogens such as toluene, dodecanol, cyclohexanol and polypropylene glycol. These particles were tested as stationary phase in high-performance liquid chromatography for the separation of polycyclic aromatic hydrocarbons in reversed-phase mode. The separation was observed even for elution 100% organic (methanol) without any participation of water fraction in the eluent composition. The influences of particles size, specific surface area and packing conditions on the separation behavior were investigated.

A framework for characterizing terminological systems.

OBJECTIVES: The notion of a terminological system (TS) is complex due to the broad range of systems, applications, and clinical domains. A uniform approach to describe the characteristics of TSs is lacking. This impedes furthering understanding, applicability, mutual comparison and development of TSs. For these reasons we propose a terminological systems characterization framework. METHODS: Relevant issues pertaining to TSs and terminology servers have been extracted from literature describing requirements and functionality of TSs. From these issues, features have been distilled and further refined. A categorization has been developed to provide a convenient arrangement of these features. RESULTS: The framework distinguishes between application-dependent and application-independent features of TSs. Definitions are provided for measures of content coverage, which was identified as the only application-dependent feature. Application-independent features are categorized along two axes: their respective type of TS and the particular element within that system, i.e. the formalism, the content, or the functionality. For each feature we provide an explicit question, the answer to which yields a feature value. The framework has been applied to SNOMED CT and the CLUE browser. CONCLUSIONS: We present and apply a framework to support a feature-based characterization of terminological systems. Standardized methods for content coverage studies reduce the effort of assessing the applicability of a TS for a specific clinical setting. A two-axial categorization provides a convenient arrangement of the large number of application-independent features. Application of the framework increases comparability of terminological systems. This framework may also help TS developers determine how their system can be improved.

Methods for evaluation of medical terminological systems--a literature review and a case study.

OBJECTIVES: The usability of terminological systems (TSs) strongly depends on the coverage and correctness of their content. The objective of this study was to create a literature overview of aspects related to the content of TSs and of methods for the evaluation of the content of TSs. The extent to which these methods overlap or complement each other is investigated. METHODS: We reviewed literature and composed definitions for aspects of the evaluation of the content of TSs. Of the methods described in literature three were selected: 1) Concept matching in which two samples of concepts representing a) documentation of reasons for admission in daily care practice and b) aggregation of patient groups for research, are looked up in the TS in order to assess its coverage; 2) Formal algorithmic evaluation in which reasoning on the formally represented content is used to detect inconsistencies; and 3) Expert review in which a random sample of concepts are checked for incorrect and incomplete terms and relations. These evaluation methods were applied in a case study on the locally developed TS DICE (Diagnoses for Intensive Care Evaluation). RESULTS: None of the applied methods covered all the aspects of the content of a TS. The results of concept matching differed for the two use cases (63% vs. 52% perfect matches). Expert review revealed many more errors and incompleteness than formal algorithmic evaluation. CONCLUSIONS: To evaluate the content of a TS, using a combination of evaluation methods is preferable. Different representative samples, reflecting the uses of TSs, lead to different results for concept matching. Expert review appears to be very valuable, but time consuming. Formal algorithmic evaluation has the potential to decrease the workload of human reviewers but detects only logical inconsistencies. Further research is required to exploit the potentials of formal algorithmic evaluation.

Comparison of methods for evaluation of medical terminological systems.

The importance of terminological systems (TS) for the medical domain is widely recognized. The usability of such a system depends primarily on its content. We have designed four methods to evaluate the content of TS and applied them in a case study

Usability of expressive description logics--a case study in UMLS.

Research in (medical) terminological knowledge representation is showing an increased interest in the family of Description Logics (DLs), as they allow for automatic reasoning. This interest is driven by an increase in demands on the quality of and reasoning ability with medical terminological knowledge. Recent advances in Computer Science have demonstrated the computational decidability and empirical tractability of quite expressive DLs. The question arises whether this expressivity is usable and useful. This paper motivates and describes an exploratory study to address this question by examining the surplus value of individual DL constructors based on an investigation of UMLS terms. Our study indicates that the disjunction and negation operators comprise very valuable extensions to current DLs. The impact of formalization depends on the involved semantic type; "Injury and Poisoning" is one of the semantic types in which a large portion of concepts will benefit from the extension.

Software engineering in medical informatics: the academic hospital as learning environment.

In 2001, the revised course Software Engineering has been implemented in the Medical Informatics curriculum at the Academic Medical Center, Amsterdam. This 13 weeks, full-time course consists of three parts: internship, theory and project. All parts are provided in problem-oriented manner with special attention for relevant skills such as project management, documentation and presentation. During the internship, students observe how health care professionals at several hospital wards work and how information supply is organized. In the theory part, students study concepts and methods of software engineering by means of case descriptions and self-directed learning. During the project, they apply their acquired knowledge to an observed, clinical information problem and complete several stages of the software engineering process. Evaluation by inquiry showed that, compared to other courses, students spent more time, and distributed their time more evenly, during the whole period of the course. In conjunction with theory, a combination of internship and project in a hospital seems to provide a surplus value compared to a practical in a computer laboratory. The integration of software theory, clinical practice and problem-based approach, contributed to the enthusiastic, intensive and realistic way students learned in this important topic that might be chosen as a future profession.

Renal replacement therapy in Europe: the results of a collaborative effort by the ERA-EDTA registry and six national or regional registries.

BACKGROUND: In June 2000 a new ERA-EDTA Registry Office was opened in Amsterdam. This Registry will only collect core data on renal replacement therapy (RRT) through national and regional registries. This paper reports the technical and epidemiological results of a pilot study combining the data from six registries. METHODS: Data from the national renal registries of Austria, Finland, French-Belgium, The Netherlands, Norway, and Scotland were combined. Patients starting RRT between 1980 and 1999 (n=57371) were included in the analyses. Cox proportional hazards regression was used to predict survival. RESULTS: The use of different coding systems for ESRD treatment by the registries made it difficult to merge the data. Incidence and prevalence of RRT showed a continuous increase with a marked variation in rates between countries. The 2-, 5- and 10-year patient survival was 67, 35 and 11% in dialysis patients and 90, 81 and 64% after a first renal allograft. Multivariate analysis showed a slightly better survival on dialysis in the 1990-1994 (RR 0.94, 95% CI 0.90-0.98) and the 1995-1999 cohort (RR 0.88, 95% CI 0.84-0.92) compared to the 1980-1984 cohort. In contrast, there was a much greater improvement in transplant-patient survival, resulting in a 56% reduction in the risk of death within the 1995-1999 cohort (RR 0.44, 95% CI 0.39-0.50) compared to the 1980-1984 cohort. CONCLUSIONS: This study provides support for the feasibility of a "new style" ERA-EDTA registry and the collection of data is now being extended to other countries. The improvement in patient survival over the last two decades has been much greater in transplant recipients than in dialysis patients.

Evaluation of DICE, a terminological system for intensive care.

Evaluative research and the introduction of the Patient Data Management System to support care have increased the need for structured and standardized registration of diagnostic information in Dutch intensive cares (IC). To this end a terminological system to describe diagnoses is needed. A terminological system is a system that denotes terms to concepts in a domain based on specifications of these concepts. During the last two years we have developed DICE (Diagnoses for Intensive Care Evaluation), a terminological system application which includes knowledge about the IC diagnoses domain, such as the anatomical localization, the pathophysiology and the etiology. This paper briefly describes the design of DICE and focus on the preliminary evaluation of DICE. DICE was evaluated on the basis of 126 diagnoses collected. The knowledge modeller as well as the intensivists judged DICE positively. However, there were some points for improvement. The knowledge modeller observed a problem in modelling dependencies between qualifiers of a concept and the intensivists observed some gaps in the knowledge base and were critical about the current interface to compose plural operative procedures.

Analysis and design of an ontology for intensive care diagnoses.

Information about the patient's health status and about medical problems in general, play an important role in stratifying a patient population for quality assurance of intensive care. A terminological system which supports both the description of health problems for daily care practice and the aggregation of diagnostic information for evaluative research, is desirable for description of the patient population. This study describes the engineering of an ontology that facilitates a terminological system for intensive care diagnoses. We analyzed the criteria for such an ontology and evaluated existing terminological systems according to these criteria. The analysis shows that none of the existing terminological systems completely satisfies all our criteria. We describe choices regarding design, content and representation of a new ontology on which an adequate terminological system is based. The proposed ontology is characterized by the explicit and formal representation of the domain model, the metaspecification of its concepts, the vocabulary to define concepts and the nomenclature to support the composition of new concepts.

Problems with integrating legacy systems.

The economic and organizational impact of imposing state-of-the-art technology to the large number of proprietary legacy systems operational in most hospitals requires integrated clinical professional workstations to provide flexible encapsulation mechanisms for these systems rather than reengineering these systems to this new technology. In this paper the implications of different input/output and translation models of legacy systems for their integration into a clinical workstation is described. Examples of legacy systems that have been integrated in the HERMES clinical workstation are presented as examples of the range of difficulties one might encounter. The features that an integrated workstation should offer for integrating a broad range of legacy systems are also addressed in this paper.